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In 1996 the European Association for Aviation Psychology (EAAP) held its 22nd 
Conference in Sabaudia, Italy. This conference was planned as a basic introduction 
into the five main areas of Aviation Psychology: Human Engineering, Selection, 
Training, Psychological Counselling/Intervention and Human Factors Accident 
Investigation/Prevention. The contributions to the conference were later published 
to form a basic textbook on aviation psychology. I myself was the editor of this 
book Aviation Psychology: A science and a profession (Ashgate, 1998). Since 
1996 aviation psychology in Europe was quite active and a couple of studies and 
developments came out with interesting results. For this new book Aviation 
Psychology: Practice and Research I have collected papers which have been 
written in the last five years. Many of the articles were originally presented at the 
EAAP conferences in Vienna (1998), Crieff (2000) and Warsaw (2002). All these 
conferences printed their proceedings in the usual way; but several authors insisted 
on their material being published in a conventional book. These authors hopefully 
will be satisfied by this book. Other sources for this book are basic research and 
development reviews and otherwise unpublished material. The book gives a good 
overview with respect to aviation psychology activities in Europe, but does not 
claim to be exhaustive. Just as my first book on Aviation Psychology, this book 
comprises a significant contribution from Australian aviation psychology in part 6 
‘Accident Investigation and Prevention’. Traditionally, a close relationship exists 
between Rob Lee and Brent Hayward from Australia on one side and EAAP on the 
other side. For years these two experts were running the very successful EAAP 
course ‘Human Factors Accident Investigation and Prevention’ together with the 
former EAAP president Kristina Pollack. Brent Hayward is a contributor to this 
book again. 

I have worked for more than 33 years in the field of aviation psychology. 
During all these years I had some excursions into deep sea diving and manned 
space flights. These two areas were interesting but one always suffered from low 
numbers of subjects if one wanted to do systematic scientific research. Contrary to 
this situation aviation as a well established system is large enough that as a human 
factors researcher one always finds a sufficient number of persons for 
investigation. Thus research results can be based on representative samples. To my 
experience people in aviation are well aware of the importance of human factors 
since the consequences of its neglect are often quite obviously seen in incidents 
and accidents. This situation helps to achieve acceptance of the system for proper 
human factors research. 
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After my years of work in aviation psychology I am happy to serve the field 
by producing this book. Its content fits well with my first book on aviation 
psychology. It follows the same structure regarding the main domains of aviation 
psychology, but the papers are not repetitions or extensions of the earlier articles. 
They add substantial new information to what was published before. In Part 1 
‘Human Engineering’ the reader will learn about human centred automation, 
cooperative and assisting flight guidance systems, and human factors in aircraft 
certification. Part 2 ‘Occupational Demands’ will present results regarding the 
occupational demands of pilots and air traffic controllers. Part 3 ‘Selection of 
Aviation Personnel’ presents information on computerised aptitude testing, 
personality evaluation and assessment centre approaches in the recruitment of 
aviation personnel. In this part the reader will also find two contributions on the 
economic value of psychological selection. The reported cost-benefits by selection 
are extremely convincing and again underline the importance of this traditional 
field of aviation psychology. In Part 4 ‘Human Factors Training’ it will become 
clear that psychological concepts have found their way via crew resource 
management training into the official world of flight crew licensing. A couple of 
papers deal with the question of how crew resource management skills can be 
objectively assessed by systematic rating tools so that training effects become 
manifest as required by the aviation authority. The demonstration of training 
effects is also described in a chapter on the validation of crew resource 
management training. Part 5 ‘Clinical Psychology' gives information how to deal 
with problem cases either by psychological evaluation (diagnosis) or by treatment 
of post-traumatic stress effects which might not only happen to core personnel such 
as pilots or controllers, but also to other people who might be involved in severe 
incidents or accidents such as ground staff, passengers or even professional aid 
personnel. A third chapter deals with the psycho-physiological evaluation of stress 
effects under regular working conditions (workload measurement). It offers a 
convincing approach about how to integrate different physiological parameters into 
meaningful reaction patterns. Part 6 ‘Accident Investigation and Prevention’ refers 
to the systematic evaluation of incidents or accidents in aviation. The two chapters 
in Part 6 are complementary to each other, since one is concerned with human 
error in air traffic control while the other gives an example for a systemic approach 
in analysing errors of cockpit crews embedded in their organisational environment. 
Both contributions emphasise that accident investigation methods developed in 
aviation can be adapted to other industries for the sake of higher system safety. 

During the preparation of the book I realised that in many areas experts are 
using abbreviations as if they are normal words from the dictionary. A latent 
pretension exists that these abbreviations should be understood by others. This is a 
habit which should be avoided if one tries to reach readers who are not from a 
specific field. In this book I followed the classical rule: If an abbreviation is 
mentioned the first time in a chapter it has to be written in full. On the other hand 
these short forms will be also mentioned in the list of abbreviations. This should 
help readers who are stepping into the running text. 

I would like to emphasise that the work published in the various chapters in 
this book could only be performed with the support of international European 
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authorities (European Community DG TREN and EUROCONTROL) or by 
national organisations (DFS, DLR, IMASSA, NLR, Sofreavia), airlines (Alitalia, 
British Airways, KLM, Lufthansa) and universities (University of Aberdeen). All 
these organisations are interested in enhancing the safety and efficiency of the air 
transportation system and regularly invest all kind of efforts to reach this goal. 

I would like to thank all authors for their work; but finally all authors should 
realise that the hard work of preparing the Camera Ready Copy of the book was 
solely in the hands of Karen Thomas (DLR - Aviation and Space Psychology, 
Elamburg). She did an excellent job in shaping the raw material into a nicely 
looking professional outfit. Thank you, Karen Thomas! 

A couple of chapters received a brush up in English. This was very well 
performed by Diplom-Psych. Mitra Schiimann-Sen. Nevertheless, the reader 
should also realise most authors are not native English speakers. One should be 
tolerant with their international English. The content of the book should count. I 
hope the readers who are interested in aviation psychology will take this book as a 
useful source of information. 



This page intentionally left blank 



List of Abbreviations 


(A)AF 

AAIC 

AASD 

AATP 

AC 

ACP 

ACSWG 

ADVISE 

AFCS 

AFIRA 

AFMS 

AFOQT 

AFQ 

AFSG 

AHMI 

AIS 

AMANDA 

AMC 

AMS 

ANOVA 

ANSP 

AOC 

AOT 

AQP 

AS 

ASC 

ASDE 

AS I 

ASVAB 

ATC 

ATCO 

ATHEANA 

ATM 

ATMOS 

ATP(L) 

ATT AS 

ATS 

ATSB 


(Army) Air Force 

Aircraft Accident Investigation Commission 
After Action Stress Debriefing 
Anti-Airsickness Training Program 
Assessment Center (1) or Advisory Circular (2) 
Altitude Separation to Closest Point 
Air Crew Selection Working Group 
Advanced Visual System for Situation Awareness 
Enhancement 

Automatic Flight Control System 
Automated Fault Identification and Recovery 
Agent 

Advanced Flight Management System 

Air Force Officer Qualifying Test 

Acronym for Follow-up Questioning 

Air Force Sub-Group 

Airborne Human Machine Interface 

Aeronautical Information Services 

Automation and MAN-machine Delegation of Action 

Acceptable Means of Compliance 

Aeromedical Section 

Analysis of Variance 

Air Navigation Service Provider 

Airline Operation Centre 

Autonomic Outlet Type 

Advanced Qualification Program 

Assessment Scenario 

Aviation Safety Council 

Airport Surface Detection Equipment 

Air Safety Improvement 

Armed Services Vocational Aptitude Battery 

Air Traffic Control 

Air Traffic Controller 

A Technique for Human Error Analysis 

Air Traffic Management 

Air Traffic Management and Operations Simulator 
Airline Transport Pilot (Licence) 

Advanced Technologies Testing Aircraft System 

Air Traffic Services 

Australian Transport Safety Board 
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BARB 

BASA 

BASI 

BOS 

BP 

BS 

CAA 

CAHR 

CAIR 

CAMA 

CAMS 

CAO 

CASA 

CASSY 

CAST 

CAT 

CBA 

CBT 

CCD 

CCT 

CDG 

CDM 

CDU 

CG 

CHAID 

CISD 

CISM 

CI(T) 

CMAQ 

CNS 

CoFoR 

COOPATS 

CPL 

CRMI 

CRMIE 

CSCW 

CTA 

CWS 

D 

DAC 

DARA 

DCP 

DCT 
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British Army Recruitment Battery 

Bulgarian Aerospace Agency 

(Australian) Bureau of Air Safety Investigation 

Behaviour Observation Scales 

Blood Pressure 

Benchmark Scenario 

Civil Aviation Authority 

Connectionism Assessment of Human Reliability 

Confidential Aviation Incident Reporting 

Crew Assistant for Military Aircraft 

Cabin Air Management System 

Civil Aviation Orders 

Civil Aviation Safety Authority 

Cockpit Assistant System 

Consequences of future ATM for ATCO Selection and 
Training 

Computer Assisted Testing 
Computer Based Assessment 
Computer Based Training 
Cursor Control Device 
Complex Coordination Test 
Core Drafting Group 
Collaborative Decision Making 
Control Display Unit 
Control Group 

Chi-Square-Aided Automatic Interaction Detection 
Critical Incident Stress Debriefing 
Critical Incident Stress Management 
Critical Incidents (Technique) 

Cockpit Management Attitudes Questionnaire 

Central Nervous System 

Common Frame of Reference 

Cooperative ATS 

Crew Resource Management 

CRM Instructor 

CRM Instructor Examiner 

Computer Supported Cooperative Work 

Cognitive Task Analysis 

Common Work Space 

Deutschland / Germany 
Dynamic Airtraffic Control Test 
(Former) German Aerospace Agency 
Distance to Closest Point 
Dyadic Cooperation Test 
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DFS 

Deutsche Flugsicherung 

DGTREN 

Directorate General / Transport and Energy 

DLR 

Deutsches Zentrum ftir Luft- und Raumfahrt e. V. 

DMT 

Defence Mechanism Test 

DOS 

Disk-oriented Operating System 

DSM 

Diagnostic and Statistical Manual of Mental Disorders 

DSS 

Decision Support System 

EAAP 

European Association for Aviation Psychology 

EADS 

European Aeronautic Defence and Space Company 

EASA 

European Agency for Safety in Aviation 

EATCHIP 

European Air Traffic Control Harmonization and 
Integration Programme 

EATMP 

European ATM Programme 

EC 

European Commission 

ECAC 

European Civil Aviation Conference 

ECG 

Electrocardiogram 

EC IP 

European Convergence and Implementation Plan 

ECL 

Electronic Check List 

EEG 

Electroencephalogram 

EFIS 

Electric Flight Instrument System 

EFMS 

Experimental Flight Management System 

EG 

Experimental Group 

EMG 

Electro-myogram 

ENJJPT 

Euro NATO Joint Jet Pilot Training 

EOG 

Electro-oculogram 

ERTS 

Experimental Run Time System 

ESSAI 

Enhanced Safety through Situation Awareness 
Integration in training 

ESVS 

Enhanced and Synthetic Vision System 

F 

France 

F/E, FE 

Flight Engineer 

F/O, FO 

First Officer 

FAA 

Federal Aviation Administration 

FAR 

Federal Aviation Regulation 

FASA 

Factors Affecting Situation Awareness 

FCL 

Flight Crew Licensing 

FDP 

Flight Data Processing 

FEAST 

First European ATCO Selection Test 

FHA 

Functional Hazard Assessment 

FIS 

Flight Information Service 

F-JAS 

Fleishman - Job Analysis Survey 

FMAQ 

Flight Management Attitudes Questionnaire 

FM(S) 

Flight Management (System) 

FNPT 

Flight and Navigation Procedures Trainer 
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FOI 

FOR-DEC 

GAF 

GAFSC 

GB 

GEMS 

GFT 

GHMI 

GM 

GND 

GPWS 

HAL 

HEA 

HERA 

HF 

HFACS 

HFHWG 

HFStG 

HLG 

HMI 

HPD 

HPL 

HR 

HRA 

HRV 

HTMM 

I 

IAEA 

I AT A 

IBMP 

ICA 

ICAM 

ICAO 

ICC 

ID 

IEM 

IFALPA 

IFR 

ILS 

IMASSA 


Aviation Psychology: Practice and Research 

Fear of Flying 

Flight Operations Inspector 

Acronym: facts, options, risks (phase of consideration); 

decision, execution, check 

German Air Force 

German Air Force Stress Concept 

Great Britain 

Generic Error Modelling System 
General Failure Types 
Ground HMI 
General Manager 
Ground 

Ground Proximity Warning System 

Human Assessment Laboratory 
Human Error Analysis 
Human Error in ATM 
Human Factors 

Human Factors Analysis and Classification System 

Human Factors Harmonization Working Group 

Human Factors Steering Group 

High Level Group 

Human-Machine-Interface 

Heart Period Duration 

Human Performance & Limitations 

Heart Rate 

Human Reliability Assessment 
Heart Rate Variability 
Heterotrait-monomethod 

Italy 

International Atomic Agency 
International Air Transport Association 
Institute for Biomedical Problems Moscow 
Instrument Coordination Analyzer 
Incident Cause Analysis Method 
International Civil Aviation Organisation 
Intra-Class Correlation 
Individualism-Collectivism 
Instructional & Explanatory Material 
International Federation of Air Line Pilots 
Associations 
Instrument Flight Rules 
Instrument Landing System 

Institut Medicine Aerospatiale du Service de Sante des 
Armees 
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IMC 

Instrument Meteorological Conditions 

IP 

Instructor Pilot 

IPA 

Intelligent Pilot Assistant 

IQ 

Intelligence Quotient 

IR 

Instrument Rating 

ISO 

International Standardisation Organisation 

ITMS 

Index of Threat Management Strategies 

JAA 

Joint Aviation Authorities 

JAR 

Joint Aviation Requirements 

JAR- FCL 

JAR- Flight Crew Licensing 

JAR- OPS 

JAR- Operations 

JAR-TEL 

JAR- Translation and Elaboration of Legislation 

JT-FMAQ 

Joint Flight Management Attitudes Questionnaire 

KASOs 

Knowledge, Abilities, Skills and Other Characteristics 

KLM 

Royal Dutch Airlines 

LAME 

Licensed Aircraft Maintenance Engineer 

LCD 

Liquid Crystal Display 

LLC 

Line / LOS Checklist 

LOA 

Level of Automation 

LOFT 

Line Oriented Flight Training 

LOS 

Line Oriented Simulation 

MAU 

Modular Avionics Units 

MCC 

Multi Crew Cooperation 

MCDU 

Multifunction Control / Display Unit 

MDA 

Minimum Descent Altitude 

MEL 

Minimum Equipment List 

MMPI 

Minnesota Multiphasic Personality Inventory 

MORT 

Management Oversight and Risk Tree 

MSP 

Manpower Sub-Program 

MTC 

Multiple Task Coordination Test 

MTHM 

Monotrait-Heteromethod 

MTMM 

Multitrait-Multimethod 

NASA 

National Aeronautics and Space Administration 

NATO 

North Atlantic Treaty Organization 

NATS 

National Air Traffic Services 

ND 

Navigation Display 

NDB 

Non-Directional (Radio) Beacon 

NEO-FFI 

Neuroticism - Extraversion - Openness to experiences - 
Five Factors Inventory 

NLR 

Nationaal Lucht- en Ruimtevaartlaboratorium - National 
Aerospace Laboratory (Netherlands) 
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N/0 

Not Observed 

NOTECHS 

Non-Technical Skills (Assessment) 

NPA 

Notice of Proposed Amendment 

NT(S) 

Non-Technical Skills 

OASIS 

Occurrence Analysis and Safety Information 

System 

OECD 

Organisation for Economic Co-Operation and 
Development 

OHP 

Overhead Projector (1) or Overhead Panel (2) 

OJT 

On the Job Training 

OPQ 

Occupational Personality Questionnaire 

OPS 

Operations 

OPT 

Optical Perception Test 

PARTI 

Pilot Display of Air Traffic Information 

PATS 

Psychophysiological Assessment Test System 

PAV 

Psychophysiological Activity Vector 

PC 

Personal Computer 

PCA 

Principle Component Analysis 

PCI 

Personality Characteristics Inventory 

PD 

Power Distance 

PD/2 

Phare Demonstration 2 

PDA 

Personal Digital Assistant 

PF 

Pilot Flying 

PFD 

Primary Flight Display 

16 PF(T) 

16-Personality Factors (Test) 

PHARE 

Program for Harmonizated ATM Research in 

Eurocontrol 

PIC 

Pilot in Command 

PNF 

Pilot Not Flying 

PPL 

Private Pilot Licence 

PRF 

Personality Research Form 

PSA 

Probabilistic Safety Assessments 

PSM + ICR 

Propulsion System Malfunction plus Inappropriate Crew 
Response 

PSW 

Parasuraman-Sheridan-Wickens-Model 

PTT 

Pulse Transition Time 

PTSD 

Post Traumatic Stress Disorder 

PUS 

Permissible Unserviceability Schedule 

PVD 

Para-Visual Display 

RA 

Resolution Advisory 

RAAF 

Royal Australian Air Force 

RAF 

Royal Air France 

RCT 

Rudder Control Test 

RG(T) 

Repertory Grid (Technique) 
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RPT 

Regular Public Transport 

RR 

Riva-Rocci 

SA 

Situation Awareness 

SACHA 

Separation and Control Hiring Assessment 

SAGAT 

Situation Awareness Global Assessment 

Technique 

SC 

Space Craft 

SCL 

Skin Conductance Level 

SCR 

Skin Conductance Response 

SD 

Standard Deviation 

SDS 

Spatial Disorientation Simulator 

SFINCSS 

Simulation of a Flight of International Crews on Space 
Station 

SHAPE 

Solutions for Human Automation Partnership in 
European ATM 

SIA 

Singapore Airlines 

SIAM 

Systemic Incident Analysis Model 

SIM 

Simulator 

SLOA 

Stakeholders Line of Action 

SMGCS 

Surface Movement Guidance and Control System 

SOF 

Student Orientation Flight 

SOFREAVIA 

Societe Francaise d’ Etudes et de Realisations d’ 
Equipments Aeronautiques 

SOP 

Standard Operating Procedure 

SPAM 

Situation Present Assessment Method 

SPSS 

Statistical Package Social Sciences 

SQL 

Search and Query Language 

SSA 

Shared Situation Awareness 

STANINE 

Standard Nine (Scale) 

STF 

Selection Task Force 

SWAT 

Subjective Workload Assessment Technique 

TARMAC 

Taxi and Ramp Management and Control 

TCAS 

Traffic and Alert and Collision Avoidance System 

TCP 

Time to Closest Point 

THC(T) 

Two-Hand Coordination (Test) 

TLX 

Task Load Index 

TM 

Threat Management 

TMA 

Terminal Area 

TOM 

Test of Multiple task performance 

TQM 

Total Quality Management 

TRM 

Team Resource Management 

TSB 

Transport Safety Board 

TSS 

Temperament Structure Scales 
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UA 

Uncertainty Avoidance 

UK 

United Kingdom 

UN 

United Nations 

US 

United States 

UT 

University of Texas 

VDI 

Verein Deutscher Ingenieure / Society of German 
Engineers 

VDT 

Visual Display Terminal 

VERDI 

Verhaltensorientierte Personlichkeitsdiagnostik 
(Assessment Center Diagnostics) 

VHF 

Very High Frequency 



PART 1 

HUMAN ENGINEERING 



This page intentionally left blank 



Chapter 1 

Human-Centred Automation: 
Research and Design Issues 

Bernd Lorenz 


Introduction 

Operators particularly in the domain of air transportation (pilots, air-traffic 
controllers) need to process huge amounts of data that emerge in a highly dynamic, 
distributed, and real-time environment. Operators have to integrate this data into 
meaningful assessment with respect to their task goals and transform this into 
efficient and coordinated action. Automation technology has proven successful at 
compensating for constraints imposed by human vulnerabilities and limitations in 
information processing and at augmenting the overall system performance in terms 
of safety and efficiency. The expected increase in air travel and the resulting 
increasing demand on a more efficient use of air transportation resources (airspace, 
airports) are crucial driving forces for a comprehensive innovation of air traffic 
management and the implementation of advanced automation technology in the 
United States (Wickens, Mavor, Parasuraman, & McGee, 1998) and Europe 
(Eurocontrol, 1998). Because of the above-mentioned key characteristic of the 
working context of pilots and air-traffic controllers their job is foremost cognitive 
and collaborative. Therefore, the nature of support to be provided by automation 
should be cognitive and collaborative. In the process of automation some of this 
activity is transferred to a machine, typically a computer or a network of 
computers. This creates some sort of sharing, i.e. partitioning the activity between 
humans and machines introducing a new demand on co-ordination between both 
agents. Given the capability of today’s automation technology more and more 
aspects of cognitive and collaborative activity can principally be transferred to the 
machine. Therefore: ‘The question is no longer whether one or another function 
can be automated but, rather, whether it should be.’ (Wiener & Curry, 1980, 
p. 995). The attempt to base this decision on a careful and deliberate evaluation of 
the consequences of automation on human performance to enable optimal joint 
human-machine cognitive and collaborative performance is at the core of a human- 
centred design of automation. 
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Billings ’ Guidelines 

Changing the mix of functions that are automated and that are left with the operator 
often significantly changes the nature of the cognitive and collaborative demands 
imposed on the operator. One of the most critical changes following automation 
occurs as a result of depriving the operators of their active involvement in system 
control activity by allocating them a passive monitoring role (Wiener & Curry, 
1980; Metzger & Parasuraman, 2001a). Simultaneously, operators are required to 
detect automation failures and to resume control in the role of a back-up. There is 
sound empirical evidence that this demand is associated with serious performance 
deficiencies. Operators’ performance in detecting automation failures in a timely 
manner may deteriorate by their tendency to over-rely on automation with 
subsequent negligence to monitor the automated function, which has been referred 
to as automation complacency (Parasuraman, Molloy, & Singh, 1993; Metzger & 
Parasuraman, 2001b). But even if the automation failure is detected or is made 
salient to the operator, poor return-to-manual performance may result, which is 
also linked to operators being disengaged from actively controlling the system 
processes. Distanced from the control process operators may lose track of the 
system state or become unable to project its state into the future, hence, operators 
lose their situation awareness (Endsley, 1995; Endsley & Kiris, 1995, Endsley & 
Kaber, 1999). Finally, in the long term, operators may lack sufficient proficiency in 
their manual skills following a process of decay known to be associated with 
disuse. This may seriously limit the accomplishment of back-up or recovery duties 
when automation fails or cannot handle an off-routine situation. Over-reliance, 
reduced situation awareness, and de-skilling have been collectively termed ‘out-of- 
the-loop unfamiliarity’ (Wickens & Hollands, 2000; Wickens, 2002). 

From this follows that keeping the human in the loop has to be the key 
principle of human-centred design of automation. To put this into the full context 
of Billings’ (1991; 1997, p. 39) guiding principles, human-centred design starts 
with the premise that: 

• ‘Pilots and controllers bear the responsibility for safety of flight or traffic 
separation and safe traffic flow, respectively.’ 

Accepting this premise leads to the axiom that: 

• ‘Pilots and controllers must remain in command of their flights or air traffic, 
respectively’, 

which in turn, leads to the following six corollaries: 

1. Both operators must be actively involved. 

2. Both operators must be adequately informed. 

3. The operators must be able to monitor the automation assisting them. 

4. The automated systems must therefore be predictable. 
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5. The automated systems must also monitor the human operators. 

6. Every intelligent system element must know the intent of other intelligent 

system elements. 

In following these guidelines, however, some difficulties arise from the view 
with which we look at human vulnerabilities and proneness to error (Woods, 
Johannesen, Cook, & Sarter, 1994; cf. Dekker, 2001). In the technology-centred 
view, automation is considered, and often sold to the public, as a means to increase 
safety by bringing down the often-cited 70 to 80 % contribution of human error as 
a causing factor of aviation accidents. Apart from abetting tendencies to blame the 
human in the loop, this perspective has been criticised for the misleading 
expectation that automation should become a kind of a protecting fire-wall 
between an inherently safe technical system and the inherently unreliable human 
who has to be constrained in his/her activity. Woods et al. (1994; cf. Dekker, 2001) 
call for a view that places the origin of error inside the system that has intrinsic 
contradictions between multiple goals leading to human error as a symptom not a 
cause. Therefore, it is the human who has to solve these contradictions to create 
safety. Dekker (2001) provides a series of examples that this active creation of 
safety is accomplished at the front end of the operator shaping the use of 
automation artefacts. 

The expectation to eradicate human error by automation is also markedly 
subdued by the experience that automation itself can induce human error (Wiener 
& Curry, 1980) or that the human encounters new kinds of conflicting goals and, as 
a consequence, new opportunities for new kinds of human error such as, e.g., mode 
errors (Woods et al., 1994; Sarter & Woods, 1995). Moreover, the behaviour of 
automation may become difficult for the operator to understand and anticipate 
creating what are referred to as automation surprises (Sarter, Woods, & Billings, 
1997). This adds a further piece of mode awareness to the compound of the overall 
situation awareness requirements to be maintained by the operator or the team of 
operators. Therefore, some of the unexpected problems in the human-automation 
interaction can be better described as a co-ordination breakdown between two 
powerful partners (Sarter & Woods, 2000) rather than simply human error. 

The Airbus A300 Accident at Nagoya Airport 

The fatal accident of an Airbus A300 of China Airlines near Nagoya airport in 
1994 that killed 249 of 256 passengers and all 15 crewmembers on board involved 
such a co-ordination breakdown between the crew and the automation of this 
aircraft (AAIC, 1996). The co-pilot was performing a normal ILS approach while 
he inadvertently activated the go-around mode of the flight director, which initiated 
a nose-up pitch. In order to reacquire the glide slope the co-pilot disengaged the 
auto throttle and reduced thrust manually. After re-engaging the autopilot it 
immediately re-entered the go-around mode still selected and commanded a nose- 
up pitch of 18 degrees, which the crew tried to counteract by commanding the nose 
down elevator. This conflicted with the autoflight’s logic of the go-around mode 
and its pitch-up commands. In support of the intended go-around manoeuvre the 
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flight director activated the automated stabiliser system to trim the aircraft to 
maximum nose-up. However, the stronger the force that the pilots put on the flight 
control column to activate the elevators, the faster the stabiliser trim responded to 
counteract the nose-down effort. Forty-two seconds after selecting the go-around 
mode the autopilots were disengaged again, but the aircraft kept climbing because 
of the nose-up trim. The aircraft approached stalling speed due to the excessive 
angle-of-attack that triggered a safety function - alpha floor - that commanded 
maximum thrust on both engines, which in combination with the maximum nose- 
up trim increased the aircraft’s attitude to 52.6 degrees. Instead of trimming the 
nose down the captain disengaged the alpha floor function by retarding engine 
thrust causing the Airbus to stall at 1800 ft. 

The description of this accident demonstrates that the human-automation 
interaction was one of a fight rather than one of collaboration. The second through 
the sixth corollaries of Billings’ principles were strongly violated in that there was 
no adequate feedback of the automation’s activity to the crew rendering its 
behaviour unpredictable to them and there was no mutual knowledge of intent 
between the human and the machine agents in the system. Following these and 
other incidents and accidents pointing to some unexpected problems with advanced 
automation the majority of system developers and designers accept the necessity of 
a human-centred approach. However, as already noted, it seems easier to subscribe 
to it than to translate it into design practice. Sarter et al. (1997) state a gap between 
human-centred design intentions and actual technology-centred design practice. As 
Woods (1994, cf. Sarter et al., 1997, p. 1926) has put it ‘the road to technology- 
centred systems is paved with user-centred intentions’. Sarter et al. (1997) attribute 
the reason for the intention-practice gap to the different perspectives of developers 
of automation and the people who have to use it. Whereas the developers’ view 
focuses on proving the beneficial impact on human performance, the practitioners 
view is characterised by the complicating factors that arise under the non-standard 
circumstances in the operational context. Thus, many automated tools work well 
under standard conditions but become brittle and clumsy when some unanticipated 
situations evolve. Therefore, a close link between simulation studies in the 
laboratory and observations in the field is paramount in the pursuit of human- 
centred automation research and design. 

Human-Centred Automation Policies 

There is some debate about the meaning of the label ‘human-centred’ that will not 
be dealt with here (see Sheridan, 2000; Winograd & Woods, 1997). In this chapter 
two definitions of a human-centred design policy or philosophy are adopted, which 
are in congruence with Billings’ guidelines already mentioned. 

As to the context of air traffic management automation, Wickens et al. (1998) 
characterise this as follows: 

The choice of what to automate should be guided by the need to compensate for 

human vulnerabilities and to exploit human strength. The development of the 

automated tools should proceed with the active involvement of both users and trained 
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human factors practitioners. The evaluation of such tools should be carried out with 
human-in-the-loop simulation and careful experimental design. The introduction of 
these tools into the workplace should proceed gradually, with adequate attention to 
user training, to facility differences, and to user requirements. The operational 
experience from initial introduction should be very carefully monitored, with 
mechanisms in place to respond rapidly to the lessons learned from the experiences 
(p. 13). 

This definition conveys several meanings of human-centred design such as 
exploitation of human strength and user involvement at all stages of the design life 
cycle. 

With a focus on cockpit automation Dekker (2001) states: 

Automation policies are meant to reduce the risk of co-ordination breakdowns across 
highly automated flight decks, their aim being to match the level of automation (...) 
with human roles (...) and cockpit display formats (...) (p. 261). 

Level of automation, human role, and display formats are three constituents of 
a human-centred design policy, which will be key issues in the remainder of this 
chapter. After addressing first the issue of function allocation, which is central to 
automation design, a specific focus will be placed on the problem of finding 
appropriate levels of automation. Taxonomies of levels of automation recently 
proposed in the literature (Endsley & Kaber, 1999; Parasuraman, Sheridan, & 
Wickens, 2000) and empirical research will be reviewed. Next, by referring to 
Hoc’s (2001) theoretical elaborations around the concept of a Common Frame of 
Reference, the human, and the system collaborative role is addressed in some more 
detail. 


Human-Machine Function Allocation 

Principles 

Automation changes the allocation of functions (tasks, roles) to human and 
machine resources. Traditionally, function allocation has been the task of system 
designers. Hollnagel and Bye (2000) describe three principles upon which system 
designers typically base the decision on what function to assign to the human and 
what to the machine (see also Grote, Weik, Wafler, & Zolch. 1995). The left-over 
principle means that the human must do those tasks it is not known how to 
automate or that are too difficult to automate because the task is too complex, too 
rare, or too irregular in how it must be implemented. This is technology-centred 
automation in its strongest sense. It ignores the impact of automation on the 
operator who is simply not taken into account. Left-over tasks are often boring 
monitoring tasks such as those left over by high-level automation in process 
control industries. The operators’ competence and motivation is starved, but, 
ironically, operators are expected to perform efficiently in those difficult and 
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exceptional off-routine tasks for which the system designer did not find an 
automated solution (Bainbridge, 1983). 

One of the first more explicit function allocation principles is the Fitts list 
(Fitts, 1951) also known as the MABA-MABA list (Man Are Better At - Machines 
Are Better At). Allocating to agents those tasks that are best suited to each of them 
has become a fundamental principle of function allocation. With regard to sensing 
and perception the human is superior in the detection of minor visual, auditory, and 
other sensory stimuli and better able to extract meaningful patterns. Fluman 
memory is superior in applying efficient search strategies and recalling the 
essential parts whereas the machine is powerful in the storage and literal 
reproduction of huge amounts of data. The reasoning capability of machines is 
deductive, fast and accurate in contrast to human qualities in inductive but slow 
and more inaccurate type of reasoning. One of the major strengths of the machine 
is its ability to apply great force accurately and smoothly, which makes it ideal for 
the allocation of consistent and repetitive sensory-motor action particularly when 
this has to be coupled with short response times. 

Hollnagel and Bye (2000) characterise Fitts list as a compensatory principle of 
function allocation. It directs system design to the weak and vulnerable aspects of 
the human and aims at avoiding demands on human performance that go beyond 
human limitations. It implies two important premises. First, the demands on human 
performance must be known beforehand in some quantifiable form that enables a 
comparison with machine performance. Second, some stability over time must be 
assumed as to the demand as well as to the capabilities of the human and the 
machine, respectively. The first premise is based on the assumption that humans 
are comparable to machines. Hollnagel (1999) doubts the adequacy of this 
assumption, which he refers to as the ‘substitution assumption’ (p. 31) and, hence, 
rejects the compensatory principle. He argues that machines are driven by 
algorithms that prescribe rigid step-by-step procedures. Humans, however, do not 
perform very well in this manner. Instead, they are guided by the purpose of their 
actions and able to accomplish a task in several different ways. The substitution 
assumption is similar to what Billings (1997) refers to as the ‘redundancy of parts’ 
paradigm, which is based on the principle of decomposing complex tasks to the 
smallest replaceable microtask to be assigned to people and machines 
interchangeably. Although quantification is achieved, applying Fitts principle at 
this level must ultimately lead to designing the human out of the system since 
human qualities such as flexibility and creativity elude in the process of task 
decomposition. Therefore, Billings (1997), and even Jordan (1963) reject the idea 
of comparing the human with the machine. 

Alternatively, humans and machines should be regarded as complementary. 
For similar reasons Hollnagel (1999) even rejects the concept of function 
allocation as it focuses on the idea of dividing functions or activities between 
humans and machines. Based on the cognitive systems engineering approach 
(Hollnagel & Woods, 1983) he stipulates the need to focus on the joint system 
itself rather than on the capabilities of its components. This, he claims, directs 
system design to the more important issue of human-system co-ordination or co¬ 
operation in achieving joint goals. He contrasts the principle of function allocation 
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that corresponds to derive division of labour schemes with a principle of function 
congruence or function matching that corresponds to questions as to how functions 
distributed among components of the joint system can be optimally linked to each 
other. The issue of designing for collaboration is further outlined below. 

Yet, system designers have a magnitude of options as to which system 
functions should be automated and to what extent. Therefore, there exists a need to 
make such decisions by taking into account human performance consequences. 
This issue will be addressed in the next section. 


Levels of Automation and Human Performance 

Defining Automation 

Due to the increasing cognitive functions taken on by advanced automation, 
efficient human information-processing becomes a major role of the operator in the 
complex human-automation team. Therefore, it is necessary to define automation 
according to this role and to base the investigation of automation-induced human 
performance consequences on a human performance model. 

Automation can be defined as 

the execution by a machine agent (usually a computer) of a function that was 
previously carried out by a human (Parasuraman & Riley, 1997, p. 231). 

A somewhat more specific definition that emphasises the different functions 
that automation can take on is 

any sensing, detection, information-processing, decision-making, or control action that 
could be performed by humans but is actually performed by a machine (Moray, 
Inagaki, & Itoh, 2000, p. 44). 

This definition already explicitly adopts an information-processing view. This 
view allows envisaging a wide range of external computer support as automation. 

A Qualitative Model 

Consistent with this view Parasuraman, Sheridan, and Wickens (2000) have 
recently proposed a model for types and levels of automation as a basis for the 
evaluation of human performance consequences of automation (see Figure 1.1). 
This model is subsequently referred to as Paras uraman-Sheridan-Wickens (PSW) 
model. Two assumptions are central to this model with regard to function 
allocation. First, automation is not an all-or-none phenomenon but can vary along 
different degrees or levels of automation (LOA). Finding an adequate function 
allocation becomes a matter of choosing the proper LOA rather than deciding 
whether or not to automate a task as a whole. Second, proper LOA may differ by 
processing stage. Processing stages, i.e. types of functions, are classified along a 
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four-stage sequential human performance model: (1) Information acquisition, (2) 
information analysis, (3) decision and action selection, and (4) action 
implementation. Stage 1 automation refers to any kinds of support in attention 
guiding, cueing, highlighting, or filtering information from the complex 
environment, which are typically described as automatic alarm or alerting systems. 
Support by new sensor technologies such as infrared or near-range radar sensors to 
enhance vision (Hecker & Suikat, 2000) can also be categorised as stage-1 
automation. Stage 2 automation supports the integration of information to allow 
meaningful inferences as to the system state. Examples are technologies of sensor- 
data fused with terrain-data bases (Krebs & Sinai, 2002; Meyer, 1998) or display 
integration (Bennett & Flach, 1992). Stage 3 automation relates to support in 
finding and selecting an appropriate course of action to achieve system goals. This 
technology belongs to the domain of expert systems. Stage 2 and stage 3 
automation define a class of automated systems that are characterised as decision 
support systems (DSS). Examples are automated tools for conflict detection (stage 
2) and resolution (stage 3) in air traffic control such as the Computer-Oriented 
Metering, Planning, and Advisory System (Volckers, 1991) or the User Request 
Evaluation Tool (Brudnicki & Me Farland, 1997). Finally, stage 4 automation 
supports the execution of actions selected in stage 3. 


Stages of 1 iifurni aliu n Processing 


1 


high 


In form atinn 
A cquisltion 

M ach ine 


low 



3 

Decision/ 


4 

A ctin n 


Information 
Analvsis Action selection implementation 


M a ch ine 


Human 


M ach ine 


Human 


Type of Automation 


M achine 


Human 


Figure 1.1 Qualitative model to determine the level of automation with 

regard to four basic human information - processing functions 
(adopted from Parasuraman et al., 2000) 

The PSW model acknowledges the fact that automation does not merely 
supplant human function but also changes the nature of human activity and 
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imposes new co-ordination demands for the human operator. Therefore, the human 
performance consequences of any automation solution are the primary focus of the 
model. Parasuraman et al. propose an iterative procedure of design evaluation 
using measures of human performance consequences such as mental workload, 
situation awareness, complacency, skill degradation, etc. as primary evaluation 
criteria. After initial types and levels of automation have been established 
secondary evaluative criteria are applied taking automation reliability and the costs 
associated with decision and action outcomes into account so as to establish the 
potential impact of performance consequences identified during the primary 
evaluative cycle on overall system performance. It has to be shown whether 
beneficial impacts of automation will unfold, which is unlikely if automation is not 
sufficiently reliable. Second, a potential impact of negative performance 
consequences, e.g. a loss of situation awareness associated with a certain type and 
level of automation, may be acceptable if errors that result from degraded situation 
awareness are not critical for system goals. The secondary evaluative cycle, 
therefore, is a risk assessment subjected to the findings of human performance 
consequences. LOA are not specified on an explicit scale. Parasuraman et al. 
propose, however, to represent the decision and action selection stage by the 10- 
point Sheridan-Verplank scale (Sheridan, 1992) depicted in Table 1.1. However, 
this scale does not only address decision functions but also functions of the action 
implementation stage. By avoiding being more precise as to the specification of 
LOA the PSW model is not bound to a specific application domain of automation. 

Table 1.1 Levels of automation of decision-making and action 

implementation functions: The Sheridan-Verplank scale 
(Sheridan, 1992) 


High 

10 

The computer decides everything, acts autonomously, 
ignores the human 


9 

Informs the human only if it, the computer, decides to 


8 

Informs the human only if asked 


7 

Executes automatically, then informs the human 


6 

Allows the human a restricted veto time before automatic 
execution 


5 

Executes a suggestion if the human approves 


4 

Suggests one alternative 


3 

Narrows the selection down to a few 


2 

The computer offers a complete set of decision/action 
alternatives 

Low 

1 

The computer offers no assistance, human must take all 
decisions and actions 


Some Empirical Findings on Types and Levels of Automation 

There is evidence that automation of decision-making functions, in particular, are 
associated with potential negative consequences of out-of-the-loop unfamiliarity 
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mentioned at the beginning of this chapter. Therefore, engaging the operator in 
decision processes appears to be critical for keeping him/her in the loop. Crocoll 
and Coury (1990) studied automation in the context of a simulated air-defence 
task. They compared two types of automation. The first provided only status 
information on a target. This can be seen as information automation affecting stage 
1 and 2 information processing. The second type involved automatic 
recommendation to engage a specific target, thus, involved decision automation 
affecting stage 3 information processing. When automation was reliable both types 
of automation resulted in superior performance relative to a manual control 
condition. Differences between types were not found. When automation became 
unreliable, however, target detection performance was worse with decision than 
with information automation. 

Sarter and Schroeder (2001) studied two implementations of an automatic aid, 
a status and a command display, that support handling of in-flight icing. Pilots in 
the status conditions received information about the location of the ice accretion, 
whereas pilots in the command conditions were given recommendations regarding 
an appropriate action (power setting, flap setting, and pitch attitude). Consistent 
with Crocoll and Coury, they found that accurate information from either types of 
automation resulted in improved handling of the icing encounter relative to 
baseline with no support. Performance with both displays dropped below baseline 
when the information was inaccurate and this performance loss was greater in the 
group supported with the command display. Rovira, Zinni, and Parasuraman 
(2002) and Rovira, McGarry, and Parasuraman (2002) obtained similar results 
using the corresponding variations of automation implemented in the Multi 
Attribute Task-Battery (Comstock & Amegard, 1992) and a simulated military 
command and control task, respectively. 

Endsiey and Kiris (1995) specifically addressed the impact of gradual 
increases of automation support of decision-making on performance of a 
comparatively simple navigational problem-solving task. They applied a five-level 
taxonomy of automation to categorise a simulated expert system that ranged from 
entirely (1) manual through (2) Decision Support, in which the participant received 
system recommendations, (3) Consensual Artificial Intelligence, in which the 
system decided and acted upon participant consensus, and (4) monitored, in which 
the system decided and acted allowing participant veto to (5) full automation. The 
experimental evaluation involved probe trials during which automation completely 
failed and required participants to resume manual control. The interesting finding 
was that prior to automation failures level-2 situation awareness, which is 
‘understanding the situation’ according to Endsley’s (1995) model, was best 
maintained under the manual condition and poorest under full automation with 
intermediate levels of automation corresponding to an intermediate level of 
situation awareness. As a consequence, operators supported with full automation 
encountered out-of-the-loop performance problems upon a breakdown of the 
expert system. This was expressed in longer decision times as compared to 
operators who previously performed the task manually. 

Later Endsiey and Kaber (1999) proposed a more elaborate level-of- 
automation taxonomy. Consistent with the concept of automation types of the PSW 
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model the Endsley-Kaber model differentiates four information-processing roles, 
which can be performed by the human, the computer, or by both agents 
simultaneously. They derived a discrete 10-level scale from the total of 81 (3 to the 
power of 4) possible combinations of categories. The scale and associated LOA 
labels are given in Table 1.2. Endsley and Kaber (1999) applied this taxonomy to 
examine the hypothesis that intermediate LOA improved performance by 
moderating mental workload while simultaneously maintaining situation 
awareness. They used a more complex and dynamic task but the same procedure as 
Endsley and Kiris (1995) of introducing a complete automation breakdown into 
normal operation with reliable automation support. Benefits during normal 
operation were largest when task implementation functions were automated, 
however, stripping away these functions was also most detrimental to performance 
when automation failed. Unfortunately, this study could not confirm the 
straightforward inverse relationship between maintenance of situation awareness 
during normal operation and the amount of out-of-the-loop performance problems 
during automation breakdown observed by Endsley and Kiris (1995). Low-level 
automation produced superior return-to-manual performance, however, situation 
awareness probed by the Situation Awareness Global Assessment Technique 
(SAGAT) was higher under higher LOA. Endsley and Kaber speculate that higher 
LOA freed resources that enabled participants to better prepare for queries of 
situation awareness. 

Table 1.2 Endsley-Kaber taxonomy of levels of automation (Endsley & 

Kaber, 1999) 
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Lorenz, Di Nocera, Rottger, and Parasuraman (2002) studied LOA effects of 
automated fault-management on operator performance using a complex, dynamic 
process control task, the Cabin Air Management System (CAMS; Hockey, Wasted, 
& Sauer, 1998). CAMS simulates some generic features of a spacecraft’s life 
support system in a simplified manner. Five subsystems (02, cabin pressure, C02, 
temperature, humidity) are kept within target states by autonomous controllers. The 
operator is required to monitor the functional efficiency of the control systems and 
to intervene in case of system faults by stabilising the system, diagnosing, and 
repairing the fault. In the extended AUTO-CAMS version, intelligent decision 
support was implemented by simulating a model-based reasoning agent 
(Automated Fault Identification and Recovery Agent, AFIRA) that provided fault 
management (FM) advisories at three different levels of automation (LOA). The 
advisory at low LOA, the Guide mode, took the form of a computerised fault 
finding or troubleshooting guide. At medium LOA, the Advisor mode, the advisory 
system had sensory access to the controlled system and provided model-based fault 
identification along with a suggested series of steps to manage the fault. At high 
LOA, the Delegate mode, the system additionally offered to implement these steps 
subject to operator veto. On the Sheridan-Verplank scale mentioned above (see 
Table 1.1) these LOA define the levels three, four, and six. The empirical 
evaluation of human performance consequences involved the experimental 
paradigm used by Endsley and Kiris (1995) and Endsley and Kaber (1999) by 
introducing a catastrophic failure of the simulated reasoning agent in 
approximately 10 % of the trials. 

As expected, performance improved when support was reliable. According to 
the findings of Endsley and Kiris (1995), it was expected that the higher the LOA 
the more performance should decrease upon automation failure. The reason for this 
was hypothesised in the higher amount of cognitive disengagement of operators 
working at higher LOA prior to the failure. Contrary to this expectation, best 
performance in terms of shortest fault identification was found when AFIRA acted 
at the highest Delegate LOA. Intermediate performance occurred at Guide LOA, 
and worst performance at the intermediate Advisor LOA. This pattern is illustrated 
in the upper panel of Figure 1.2. The difference between Advisor and Delegate 
LOA was significant (p < .05). Thus, the characteristic out-of-the-loop pattern of 
poor fault management was not found at the higher Delegate LOA as predicted but 
at the intermediate Advisor LOA. Based on an analysis of the operators’ 
information sampling and control intervention strategies a distinct difference 
between support with Advisor and Delegate LOA was found. During reliable 
support the following pattern was observed (see Figure 1.3). Whereas operators at 
Advisor LOA significantly (p < .05) reduced system-state inspection, operators at 
Delegate LOA significantly (p < .01) reduced control interventions. During AFIRA 
failure no between-LOA differences were found neither with regard to the amount 
of control interventions nor to system state inspections. Most probably, these 
different shifts in the pattern of system state inspection and control intervention 
during reliable support must be linked to the differences in fault identification 
during support failure. Thus, disengagement from controlling lower system 
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processes during reliable support in the Delegate mode appeared to preserve better 
situation awareness and hence efficiency at the higher supervisory control level. 



Normal Failure 


Figure 1.2 Duration of fault identification (upper panel) and timing error of 
mode switching from fault-mode back to normal-mode settings 
(lower panel) as a function of normal vs. failed support by the 
expert agent AFIRA separated for three different levels of 
automation (LOA) of AFIRA 

However, disengagement from control activities at lower system level was 
likewise associated with performance costs albeit in a less important secondary 
task. Participants had to switch back system configurations from failure to default 
settings after repair has been achieved. This mode switch was due exactly one 
minute after fault repair had been initiated and participants had to monitor the 
system clock to timely initiate this action. This secondary task was automated at 
Delegate LOA but remained with the participants at the other two LOA. The 
results (see bottom panel of Figure 1.2) revealed that the average timing error was 
lowest under Delegate LOA during normal operation. This is a somewhat trivial 
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benefit of automation as AFIRA perfectly timed the mode switch and the deviation 
from a zero timing error resulted from those few trials when operators vetoed 
against the correct diagnosis of AFIRA and subsequently disengaged the agent. 
More interesting was the fact that timing the mode switch at Delegate LOA 
became worst during retum-to-manual performance in case of AFIRA failure 
resulting in a significant LOA by AFIRA operation (normal vs. failure) interaction 
effect (p < .05). This effect can be regarded as a result of an automation-induced 
loss of the cognitive skill most likely involved in task switching and scheduling. 
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Figure 1.3 Average total number of control interventions (upper panel) and 
average percentage time displays were activated for system state 
inspection (lower panel) as a function of normal vs. failed support 
by the expert agent AFIRA separated for three different levels of 
automation (LOA) of AFIRA 

To explain both return-to-manual effects of, first, the faster fault identification 
and, second, the less accurately timed mode-switch observed at Delegate in 
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comparison to Advisor LOA, requires to take the two control loops of AUTO¬ 
CAMS into account. Operators at Delegate LOA disengaged from recovery 
implementation actions and, therefore, were out of the lower-level inner-control 
loop, whereas operators at Advisor LOA disengaged from fault assessment 
processes and, hence, were out of the higher-level supervisory control loop. 
Impaired operator performance in terms of return-to-manual deficits occurred at 
both LOA in behavioural functions specifically linked to the respective loops, i.e. 
fault identification time linked to the supervisory control loop and timing the mode 
switch back to normal settings linked to the inner loop. This suggests that active 
manual involvement in FM activities per se does not guarantee maintenance of 
performance in the role as a back-up of failed automation. 

Results contradicting the hypothesis that active involvement in the control of a 
task always supports maintenance of situation awareness have recently been 
reported by Jentsch, Barnett, Bowers, and Salas (1999) in human-human role 
allocation. They analysed over 300 civilian incidence reports and found a link 
between loss of situation awareness and aircrew role assignment. It turned out that 
loss of situation awareness occurred more frequently, first, when the captain was at 
the controls than when it was the first officer, second, that the pilot flying was 
more likely to lose situation awareness than the pilot not flying. They attribute this 
finding to the twofold burden of captains when engaging in active flight control 
while simultaneously maintaining the ‘big picture’ needed to fulfil their role in 
setting global goals and elaborating problem solving strategies. This explanation is 
consistent with the one stressed above for the LOA findings made with the AUTO¬ 
CAMS micro-world. It is, thus, not surprising that Billings (1991) regards the 
major benefits of automation for pilots in the cockpit by relieving this twofold 
burden. ‘Automation can lighten this burden on pilots, first by relieving them from 
the burden of inner-loop control, second by providing integrated information, and 
third by allowing them to manage at a higher level’ (p. 17). 

A similar phenomenon in role assignment was also observed in the ATC 
domain. Vanderhaegen, Crevits, Debernard, and Millot (1994) compared different 
implementation modes for dynamic task allocation between the human controller 
and a conflict resolution tool to support en-route air traffic control. In an explicit 
mode, the human air traffic controller managed the task allocator through a dialog 
interface. So the controller decided to allocate the task either to himself or to the 
tool. In the implicit mode, task allocation was managed automatically based on 
some rules that ensured that easy tasks were allocated to the tool when the overall 
task demand of the controller was high. Controller performance was best in the 
implicit mode, although the controllers subjectively preferred the explicit mode. 
The problem with the explicit mode apparently was the increased workload 
imposed on the controller by accomplishing the additional task of making a task 
allocation decision. A subsequent study (Lemoine, Debernard, Crevits, & 
Millot,1996; Hoc & Lemoine, 1998) revealed that the problem with the explicit 
mode was not only workload but also a conflicting mismatch in role assignment 
similar to the one described above. This time, the explicit mode was compared 
with an assisted explicit mode in which the planning controller was assigned the 
role of the task allocator who was supported by automatic advisories. Performance 
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evaluation was in favour of this latter mode, apparently because task allocation 
management was a strategic task for which the planning controller was better 
suited than the more tactically engaged executive radar controller. 

Summary 

Human performance oriented taxonomies of levels of automation such as the PSW 
or the Endsley-Kaber model provide an integrated view on several kinds of 
intelligent automation in support of one important human role, which is 
information processing. These models help to supply the pursuit of both 
automation policies mentioned above with empirical data. Accumulated evidence, 
so far, highlight the particular importance of the decision and action selection role 
for which the human seems to be best suited. In applying these models, however, at 
least three aspects appear critical. 

First, both LOA frameworks do not explicitly address the hierarchical and 
multi-loop characteristic that is prominent in many complex human-automation 
systems. The findings with the AUTO-CAMS micro-world have shown that 
operators can be distanced from the supervisory control loop by engaging them in 
lower-level inner-loop control. Relieving them from this burden, however, was 
also associated with out-of-the-loop performance problems. Disengagement from 
decision-making functions with either loop control was central to both types of 
performance problems. These point to the presence of a trade-off to be solved by 
system design by balancing the cost and benefits of selecting LOA across more 
than one control loop. The search for an optimal intermediate degree along an 
ordering of levels of automation does not make much sense in this context. The 
distinction of abstraction levels (Rasmussen, 1983) orthogonal to automation types 
may be a useful extension for the PSW model. 

Second, by focusing on optimal intermediate LOA or by deliberately selecting 
what to automate in terms of information-processing functions, there is an implicit 
notion that system design should prevent under- as well as over-automation. This 
view disregards the importance of the collaborative role of both intelligent agents 
that results from how they interact with each other. Thus, the question of what 
function should be automated and to what degree must be supplemented by 
addressing the question of how to co-ordinate and communicate the individual 
human and machine agents’ activity during function execution. Similarly, Norman 
(1990) points out that the problem is not over-automation but inappropriate 
feedback. However, it has to be emphasised that activities devoted to coordination 
and communication are also workload drivers and consume time. This fact 
represents a potential negative cost factor that is difficult to predict during the 
design of a new system. This may become particularly important for decisions with 
severe time constraints, e.g. a rejected take-off after the critical Vl-speed. For such 
a decision Inagaki (1999) suggests a level of automation between level-6 and -7 on 
the Sheridan-Verplank scale to ensure safety: ’The computer executes automati¬ 
cally after telling human’ what it will do. No veto is allowed for the human (p. 
158). The question whether this level is superior to level-7 (computer executes 
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automatically, then informs the human) to avoid automation surprises needs to be 
tested. 

The third point is closely related to the second. There is a consistent finding in 
the experimental studies discussed above that transitioning from routine to 
unanticipated off-routine situations is associated with a performance loss. This 
problem is known as brittleness (Billings, 1997) and is linked to limitations of the 
knowledge base of the decision aid. The amount of performance deterioration 
caused by automation brittleness may be moderated by LOA, however, finding a 
better function allocation would not solve this problem entirely because automation 
brittleness emerges from a lack of coordination or lack of feedback between the 
human and the decision aid during joint problem solving. Thus, the focus should 
not be to improve function allocation by selecting the LOA associated with the 
least amount of brittleness. Instead, automation brittleness associated with a certain 
LOA should be directly addressed by better coupling human and system resources, 
or in other words, by improving the design for more efficient human-system 
collaboration. 


Designing for Human-System Collaboration 

The Need for Common Ground 

Consider again the Airbus A300 accident at Nagoya airport. The flight director 
made several control decisions, e.g. commanding the system stabiliser to trim the 
aircraft nose-up, against conflicting elevator control inputs of the pilots. It was 
authorised to do so by the pilot’s inadvertent directive to perform a go-around. It 
did not provide feedback to the pilots as to what it was doing, why it was doing 
that, and what it was going to do next, which are the key questions to be answered 
to render an automated system to be observable and predictable (Wiener, 1989). In 
order to behave like a team player the criteria of observability and predictability 
must be supplemented by the criterion of directability (Christoffersen & Woods, 
2002). A machine agent is good at taking directions if it allows substantive human 
influence on its activities honouring the role of the human to act in a strategic role. 
Thus, with the increasing capability of automated systems the questions as to 
which information the operator should receive when and how it should be 
displayed along with questions as to how to design the human-machine 
collaborative activities become central to system design in addition to the issue of 
function allocation. 

Many authors suggest that the human-system interaction should be designed 
for optimal mutual cooperation or team play (Malin & Schreckenghost, 1992; 
Sarter & Woods, 2000; Hoc, 2001; Christoffersen & Woods, 2002). As far as 
human-human team play is concerned training to develop a better aircrew 
cooperation has been known as Crew Resource Management (Wiener. Kanki, & 
Helmreich, 1993) and has been implemented by many commercial airlines. The 
basic CRM principle can be stated as to effectively utilise all available resources 
(system and humans) to achieve mission goals (safe and efficient flight operations) 
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and to ensure that all crewmembers operate from a ‘common ground’ of the actual 
situation. The notion of common ground means that crewmembers must have 
mutually held knowledge of the situation. Each crewmember must know what the 
other crewmembers know and what their intentions are. CRM has shifted the 
emphasis from training individuals to training crews. Similarly, as already 
mentioned above, the need for such a shift in emphasis in the context of human- 
system interaction is proposed by Hollnagel and Woods’ (1983) joint cognitive 
systems approach. 

The physical locus of human-machine interaction is the Human-Machine 
Interface (HMI). Traditional ergonomics has focused on designing the more 
physical and perceptual-motor properties of the HMI surface structure (colours, 
font sizes, symbologies, input devices, etc.). However, intelligent problem-solving 
activities of computers e.g. those involved in advanced decision support systems 
assisting planning and managing arrivals, departures, and en-route flight 
trajectories in air traffic control, raise issues of a more profound cognitive nature 
that has to be approached by system design. If difficulties in human-machine 
interaction such as automation brittleness or automation surprises are to be avoided 
human and machine agents must understand each other, i.e. establish common 
ground and coordinate their action. These issues go beyond traditional HMI design. 
The challenge is to develop a design concept that facilitates the establishment of 
common ground. In the civil aviation community in the US and Europe this idea 
has been known as Collaborative Decision Making (CDM) involving at the most 
global level the three main air traffic actors, i.e. air traffic service providers, airline 
operation centres, and airport management. CDM is driven by an effort to reduce 
delays, accommodate preferences, and avoid risks by improving information 
exchange and by jointly and pro-actively coordinating the use of airspace 
resources. In the domain of information technology a new and rapidly growing 
branch of interdisciplinary research into Computer-Supported Cooperative Work 
(CSCW) has emerged (Bannon, 2001), which provides a variety of enabling 
networking technologies to build and maintain common ground in distributed 
teams. Examples are shared files, use of intelligent agents to gather information in 
large databases, communication links such as e-mail, chat lines or even virtual co- 
location. Here it is focused on design issues in the ATM domain, however not at 
the global CDM level mentioned above but at a more local level to provide one 
example of how machines can be made more cooperative (see Campion, Brander, 
& Koritsas (1998) for a design facilitating common ground in the domain of 
military command and control). 

A Framework for Human-Machine Cooperation 

Hoc (2001) proposes a framework and an architecture built upon the idea of 
common ground. He uses the term Common Frame of Reference (CoFoR). The 
aim is to convey CoFoR to all agents involved as well as to dynamically adapt 
CoFoR to the individual agents’ frame of reference if necessary. This concept has 
recently been implemented in an air traffic control project called AMANDA 
standing for Automation and MAN-machine Delegation of Action (Debemard, 
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Cathelain, Crevits, & Poulain, 2002). Hoc (2001) defines CoFoR as ‘a shared 
knowledge, belief and representation structure between the agents’ (p. 519). In an 
earlier lab-oriented stage of this research Hoc and Carlier (2002) performed an 
experiment that aimed at a description of CoFoR elements that two radar 
controllers elaborate and up-date when they have to cooperate. They observed two 
radar controllers managing the same heavy traffic together in a single sector but 
with a fixed allocation of aircraft within this sector to each controller. The 
experiment was organised such that one controller could not send any instruction to 
an aircraft allocated to the other implying the need to cooperate on shared conflicts. 
Based on verbal protocol analyses they found that ‘cooperation in action’ 
represented only 20 % of the cooperative activities, whereas the majority of 80% 
was represented by ‘cooperation in planning’. This aimed at maintaining CoFoR in 
order to detect interference between the controllers’ individual activities by simple 
exchange of information and resolving interference by agreement without long 
explanations due to time constraints. These and further results from other 
experiments highlight the importance of tight collaboration particularly in 
decision-making processes. This also points to an interesting consistency with 
empirical results following the LOA approach in that this research also emphasises 
the critical role of engagement in decision-making processes to maintain system 
state awareness. Hoc and Carlier (2002) used their empirical findings on CoFoR 
maintenance to design automation tools to be integrated in the AMANDA 
simulation platform. 
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Figure 1.4 Contents of a Common Frame of Reference (cf. Debernard et al., 
2002) 
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The contents of CoFoR are defined around Rasmussens’ (1983) decision 
ladder model that addresses the four fundamental cognitive activities of 
information elaboration, situation identification, decision-making on strategies and 
solutions, and solution implementation equivalent to types of automation functions 
of the PSW model described above (Figure 1.4). In the system architecture, CoFoR 
is implemented on a machine which is called Common Work Space (CWS). To 
cooperate, agents pass on the products of the respective activities to form the 
content of the CWS, which are information, problems, strategies / solutions, and 
commands enabling a sharing of their own frame of reference. Besides defining the 
effective contents of the CWS the design of a cooperative human-machine system 
must allow for activities that, first, up-date and control its contents to achieve 
consistence with each agent’s frame of reference and, second, for activities that 
manage the interference, i.e. coordinate the interdependent collective activities of 
agents. Therefore, inconsistencies between CWS content and agent frames of 
reference must be detected, diagnosed, and resolved. Pacaux-Lemoine and 
Debernard (2002) describe three forms of solving these inconsistencies that may be 
used by agents: negotiation, acceptance, and imposition. Negotiation aims at either 
modifying CWS content or the frame of reference of an agent on the basis of 
further explanations, which is associated with the largest costs in terms of 
demanding cognitive, communication, and time resources. Acceptance and 
imposition are reciprocal in that either an agent updates its frame of reference from 
the CWS, or opposite to that impose its frame of reference to the CWS. Debernard 
et al. (2002) further describe how this functionality has been used to integrate 
decision support tools with CWS interfaces implemented on an operational human- 
in-the-loop simulation platform. 


Concluding Remarks 

Increases in the level of automation are inevitably associated with increases in the 
complexity of automation. This requires a proportionate increase in the feedback 
automation must provide to its human partners about its activities (Norman, 1990; 
Christoffersen & Woods, 2002). Human-centred automation policies seek at 
matching the level of automation with human roles and display formats (Dekker, 
2001). This chapter has tried to elaborate the research and design issues implied to 
accomplish this policy. There is converging evidence from research on human 
performance consequences of varying levels of automation and research on human- 
human collaborative activities to maintain a Common Frame of Reference that 
decision-making processes are key in this regard. Taxonomies of levels of 
automation along with experimental research on human performance consequences 
and system design principles elaborated by Parasuraman et al. (2000) guide system 
design in human-automation function allocation by focusing on the central 
information-processing role of human operators in advanced human-automation 
systems. Finding an appropriate match with display formats means finding the 
corresponding level of cooperation which goes beyond traditional HMI design 
issues and addresses the second major human role of collaboration. Computer- 
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supported cooperative work (CSCW) will be one of the key enabling technologies 
to develop cooperative machines providing a Common Frame of Reference in 
support of collaborative decision-making. Therefore, these technologies have to 
further accommodate the cognitive, dynamic, and distributed nature of tasks in the 
aviation domain and have to keep pace with the technological development of 
advanced automation. The goal should be to pave the way to human-centred 
solutions and to avoid getting stuck in human-centred intentions. 
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11 Cost-Benefit Analysis of Pilot 
Selection: The Economic Value of 
Psychological Testing 
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12 Cost Savings: The Use of Biodata to 
Improve Selection Efficiency in Aviation 
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